1. Field of the Present Invention
The present invention relates generally to the use of body/brain signals to control a device, and more specifically it relates to systems, devices and methods employing a plurality of brain/body-generated inputs to control the multi-action operation of a controllable device.
2. Background
It can be appreciated that brain/body signaling machine systems and head-mounted display equipment have been in use for years. Brain/body signaling machine systems or products such as the Mindflex®, the Starwars Jedi Trainer® or Muse™ are typical embodiments of conventional brain/body signaling machine systems.
The main problem with these above mentioned, conventional brain/body signaling machine systems is that they only have one brain/body electroencephalography (EEG) signal that is received, processed and transmitted to a single action machine or electronic device. Such systems and devices have no methods or means to remotely operate a machine with multi-actions. Another problem with conventional brain/body signaling machine systems is that they are not able to simultaneously receive process and transmit multiple brain/body signals and accessory sensor control signals. Another problem with conventional brain/body signaling machine systems is that they are not able to process and transmit other mind-body signals such as electromyographic (EMG), electrooculographic (EOG), electroencephalographic (EEG) signals, accessory signals controls and/or biosignal data to operate a multi-action machine.
Head-mounted displays (HMD) equipped with camera devices or displays have been used for years, such as for Google's, Project Glass head-mounted display system, in Oakley's augmented reality glasses and in Apple's laser light display glasses.
A head-mounted display (HMD) is an imaging display device that a person wears on the head in order to have images or video information directly displayed in front of the eyes. HMDs are also known as near-to-eye displays. A HMD has either one or two small CRT, LCD or OLED displays with magnifying lenses and other associated optical elements. The display(s) and optics are embedded in a helmet, glasses, or a visor, which a User can wear.
HMDs may be configured as binocular HMDs for dual eye use. HMDs may also be configured as monocular HMDs for single eye use. The arrangement generally depends on the desired needs of the User.
Some HMDs can be used to view a see-through image imposed upon a real world view, thereby creating what is typically referred to as an augmented reality. This is accomplished by reflecting the video images through partially reflective mirrors, such that the real world is seen through the mirrors' reflective surfaces. The augmented reality can be combined with the stereoscopic images in various types of applications. Some examples include applications in surgery, where radiographic data, such as CAT scans or MRI imaging can be combined with the surgeon's vision. Military, police and firefighters use HMDs to display relevant tactical information, such as maps or thermal imaging data. Engineers and scientists use HMDs to provide stereoscopic views of CAD schematics, simulations or remote sensing applications. Consumer devices are also available for use in gaming and entertainment applications.
The main problem with the above mentioned conventional head-mounted displays and glasses (HMDs) is that they have no way to easily control the images or software applications which are being displayed by the HMD. Another problem with the above mentioned conventional head-mounted displays (HMDs) and glasses is that they have no way to easily control the User interface of software applications which are being displayed in the HMD. Another problem with the above mentioned conventional head-mounted displays and glasses (HMDs) is that they have no way to easily control software programs which may be used to operate and control a machine or electronic device.
Another problem with the above mentioned conventional head-mounted displays and glasses (HMDs) is that they have no way to easily control a computer or computer medium which is related, accessed or displayed by the HMD. The computer readable medium is any data storage device that can store data which can thereafter be read by a computer system. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion, such as on a computer network, the internet or internet cloud.
The advent of interactive computer systems has spurred the development of a variety of devices for enabling a User to easily input spatial coordinate data into a computer. Computer systems include some means such as a “mouse”, “joystick”, or digitizing tablet for permitting a User to move a cursor displayed on a screen and to indicate screen coordinates by depressing a button when the cursor is in the desired screen location. A typical “mouse” comprises a chassis containing a ball, with a part of the ball protruding through the underside of the chassis. When an operator moves the mouse about on a flat surface, the ball rotates. Sensors in the chassis detect the rotation of the ball about two perpendicular, horizontal axes and a computer can determine the displacement of the center of the ball from a reference point in two-dimensional space from the sensor outputs, the displacement therefore representing a two-dimensional spatial coordinate. A typical “optical” mouse operates on a pad having a reflective grid on its surface. A light emitting diode on the mouse shines a light beam on the pad which is reflected back to a light sensor on the mouse as the light passes over a grid line, enabling the mouse to detect movement. Graphics tablets employ a pad with a fine grid of wires periodically conducting electrical pulses. The operator moves a stylus about on the pad, the stylus having a sensor for detecting the pulses, and the position of the stylus on the tablet is determined by the timing of the pulses detected. An “inertial mouse” includes accelerometers for producing output signals of magnitudes proportional to its acceleration in two non-parallel directions forming a two-dimensional cartesian coordinate system. The output signal of each accelerometer is integrated to provide data from which the computer can determine the movement of the inertial mouse in two-dimensional space.
A problem with the above mentioned computer control devices are that they require a User to use their hands, fingers or a stylus to operate a “mouse” or “mouse” type controller. Another problem with these computer control devices is that they cannot be operated by the User using brain/body signals combined with accessory signal data.
Accessory Sensors such as temperature sensors, heart rate sensors, Photosensor, skin conductive sensors, barometer and tilt sensors have been known for years. Such sensors typically provide sensor data to computers so users may view numerical data acquired by the sensor or for a computer to process said data in various manners. Tilt sensors in such devices as the multipurpose controller invented by M. Parshionikar, (Publication number S20120229248 A1), Nintendo's Game Boy Advance, Nintendo's Wii controllers, Microsoft's Sidewinder, Sony's Play Station 3 and Wow Wee's robotic controllers; all use tilting sensors to control electronic devices and software programs. None of the above mentioned sensors use EEG brain/body signals together with sensor data to operate and control a multi-action machine, computer or software program by a plurality of brain-body actuated signals combined with Accessory Sensor signal data. The Parshionikar invention monitors and uses facial expressions or movements of facial muscles to control electronic devices. Parshionikar discloses sensing of facial muscles via EMG and EOG signals, but the Parshionikar invention does not utilize EEG brain/body signals which are separate from facial muscle expression signals.
While the above referenced devices may be suitable for the particular purpose to which they address, they are not as suitable for persons who to wish to operate a multi-action machine, electronic device, computer or computer application(s) in a hands-free method using brain/body signals and accessory sensor signal data. The main problem with the above mentioned conventional brain/body signaling systems, machines, HMDs and computer mouse(s) are that a multi-action machine or computer applications cannot be effectively controlled using multiple brain/body signals combined with accessory sensor signals.
In these respects, the Head-Mounted Brain-body Actuated Multi-signal Controller with Accessory Sensors according to the present invention substantially departs from the conventional concepts and designs of the prior art, and in so doing provides a head mounted system and apparatus primarily developed for the purpose of a User to operate a multi-action machine or computer application by a plurality of brain-body actuated signals with accessory sensor signals.